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Introduction 

As rural states nationwide struggle with cuts in education funding, 
declining enrollments and increased costs it becomes crucial to examine those 
schools that are successful in educating their students. These higher- 
performing schools can then be used as models for education excellence 
throughout the state. Defining higher-performing is not an easy venture and 
there are as many opinions of what constitutes higher-performing as there are 
organizations reporting on higher-performing schools. Why all this focus on 
school performance? One major factor has been PL 107-110 No Child Left 
Behind (NCLB) legislation requiring a higher level of accountability for schools 
than ever before from a federal standpoint. Schools and School Districts are 
given a report card each year and States are required to disseminate 
information on school and district performance to parents each year. With the 
increase in focus on school level accountability and the ability of the internet to 
disseminate information with ease, several organizations are beginning to study 
what variables influence school performance. This paper compares the higher- 
performing school studies of Just For The Kids (part of the National Center for 
Educational Accountability), and The Center for Education Policy Applied 
Research and Evaluation (at the University of Southern Maine) in order to open 
a discourse on what exactly determines higher-performing and what high 
schools in Maine truly are models of academic achievement. Let me begin by 
providing some background information on these organizations as stated on 
their respective websites. 

“Just For The Kids (JFTK) is part of the National Center for 
Educational Accountability. The center is a collaborative effort of 
the Education Commission of the States , The University of Texas 
at Austin , and Just for the Kids to improve learning through the 
effective use of school and student data and the identification of 
best practices. The Just For The Kids School Reports are a tool to 
help schools identify how they are performing compared to other 
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schools in the state with similar or more disadvantaged student 
populations and to learn what the highest-performing schools are 
doing to achieve academic excellence. These reports are based on 
information obtained from the state department of education in 
each state and provide an unbiased, data-based view of a school's 
academic achievement” ( http:/ / www.iust4kids.org) . 

“The Center for Education Policy, Applied Research, and 
Evaluation (CEPARE), in the College of Education and Human 
Development of the University of Southern Maine, provides 
assistance to school districts, agencies, organizations, and 
university faculty by conducting research, evaluation, and policy 
studies. In addition, CEPARE co-directs the Maine Education 
Policy Research Institute (MEPRI), an institute jointly funded by 
the Maine State Legislature and the University of Maine System. 

This institute was established to conduct studies on Maine 
education policy and the Maine public education system for the 
Maine Legislature” ( http:/ /www.usm.maine.edu/cepare/L 

At the request of the Maine state legislature, MEPRI began a study to 
determine higher-performing, cost-effective schools in Maine. The study utilizes 
Maine Educational Assessment (MEA) data, Census Data, School-level staffing 
data, and School Administrative Unit level staffing data. 

What is the significance of the “higher-performing” designation? 

Providing for education is the responsibility of each State but with 
American students falling behind their international counterparts and little or 
no improvement in achievement test scores since the Nation at Risk study in 
1983 (Peterson 2003), the federal government has stepped up pressure on the 
states to improve their schools and align curricula with the national standards. 
As noted above, NCLB requires that states hold individual schools accountable 
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for the education of all their students. Schools consistently not meeting Annual 
Yearly Progress may face dire consequences including closure or 
relinquishment of control to state or private entities. Parents have increased 
rights under NCLB including the right in some cases to request a change of 
school for their child with transportation being the responsibility of the school 
district. Schools need to improve and NCLB requires that as they initiate 
improvement plans they use scientifically based practices to improve 
educational outcomes for their students. The call for these scientifically based 
practices has led to a surge in research about what makes a school successful 
and what types of instruction are considered best practices. By examining 
those schools that are currently higher-performing we can begin to understand 
what variables are most important to student success, and using this 
information target interventions and funding in the ways most likely to improve 
student outcomes in lower-performing schools. 

Defining Higher-Performing: Varied Approaches 

When beginning a study on higher-performing schools you have to decide 
what criteria will lead to a higher-performing designation. This will be based on 
what schools are included in comparison groups and in what content areas you 
rank the schools relative to their performance on a given measure. JFTK 
examines performance by comparing each school with consistently high 
performing schools in the state that serve similar or more challenging student 
populations. Those schools that have scores within 5% of the highest scores in 
their pool are considered higher-performing. Thus an opportunity gap of 5% 
percent or less becomes the higher-performing criterion for JFTK. 

CEPARE has a series of 6 criteria that must be met for a school to be 
considered higher-performing. These criteria require that students in higher- 
performing schools score better than the state average on the MEA composite 
scale score, do better than predicted based on student and community 
variables, and do better than the state average in terms of their disaggregated 
advantaged and disadvantaged student group scores. 
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By comparing the resulting list of higher-performing high schools from 
these two organizations we can begin to better understand how to define high 
performance and use that information to help improve all schools. Those 
schools that are determined to be higher performing by both organizations may 
become models for other schools in the state. Additionally, by looking at which 
schools make each organization’s list we can begin to evaluate the value of 
each definition of higher-performing and what can be learned from each 
definition. 

Just For The Kids Higher-Performing 

To begin its higher-performing analysis Just For The Kids creates 
comparable groups of schools for all schools. These groups are formed based 
on the following requirements. 

1 . The percent of economically disadvantaged students for the school 
must be greater than or equal to the selected school or 90%. 

2. The grade size must be at least 40% the size of the grade in the 
selected school. 

3. There is at least 40% as many students tested in the grade at the 
school as are tested in the grade at the selected school. 

4. There are at least 10 tested students in the grade. 

Each school’s scores on the 2003 MEA math and reading assessments are 
compared to the comparable group in order to determine an opportunity gap 
for each school. This opportunity gap is defined as a difference between the 
percentage of the individual school’s students who meet or exceed the standard 
as compared to the weighted average percentage of the 3 top comparable 
school’s students who meet or exceed the standard (see Figure 1). Those 
schools with less than a 5% opportunity gap are considered to be strong- 
performers. In essence this creates a norm group for the school and sets a 
higher-performance criterion of being within 5% of the top three performers in 
your group. 
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Figure 1. 



■ Exceeds ■ Meets & Above 




All Tested Students Top Comparable 

Schools 

Important Note: This shows the achievement of all tested students regardless of how long they have been at the 
school. The achievement of students who have been enrolled since the school's lowest high school grade is a better reflection of a 
school's academic program and will become available when the state provides the data. 

See 'JFTK Chart Featured for more detail. 



CEPARE Higher-Performing 

CEPARE looks at higher-performing from a slightly different perspective; 
we compare each school’s 3-year average composite scale scores (average of 
reading, writing, math 8s science scores) to the average composite scale scores 
for all high schools in the state. High-performing schools are those that meet 
the following criteria: 

1 . better than state average (by 1/3 standard deviation) 

2. better than predicted (by 1/3 standardized deviation) 

3. better than state average for advantaged youth (by 1/3 standard 
deviation) 

4. better than state average for disadvantaged youth (by 1/3 standard 
deviation) 

However, since we do not have 3-year averages for criteria 3 & 4 (only two years 
of data), two additional interim criteria were created. In addition to Ideal 
criteria 1 & 2, the following criteria must be met. 

5. percent of pupils at or above Meets proficiency level is better than 
state average (by 1/3 standard deviation) 

6. percent of pupils at or above Partially Meets proficiency level is 
better than state average (by 1/3 standard deviation) 
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Any school that meets either the ideal or the interim criteria will be considered 
higher performing for purposes of this study. 

Who Makes the Lists? 

Using the Just For The Kids criterion there are 48 high schools in Maine 
that are considered higher-performing in either math or reading. Eighteen of 
those schools are strong-performers in both math and reading (having 
opportunity gaps of less than 5% in each subject). Twenty-three schools are 
strong-performers in math only and 7 schools are strong-performers in reading 
only. Using the CEPARE criterion there are 19 high schools in Maine that are 
considered higher-performing. Thirteen schools meet ideal and interim 
criterion, two schools meet ideal criterion only and four meet only the interim 
criterion. 

Table 1. JFTK Higher-Performing High Schools 



Subject Area 


Number of Schools 


Reading and Math 


18 


Reading Only 


7 


Math Only 


23 



Table 2. CEPARE Higher-Performing High Schools 



Number of Schools 

19 



Table 3. Schools Considered Higher-Performing by both CEPARE and JFTK 



JFTK Subject Areas 
and CEPARE Criterion 


Number of Schools 


Reading & Math 


5 


Reading or Math 


6 
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Sorting out the lists 

As can be seen from the various combinations and numbers of 
potentially higher-performing schools, determining the “right” higher- 
performing list is not an easy task. Perhaps, instead of trying to determine 
which list is right, a better approach would be to determine what is right about 
each list and what the drawbacks of each list are. With this approach we can 
try to learn from each definition of higher-performing and have a better 
understanding of the complexity surrounding such studies. In addition, by 
pooling the data we may be able to define a highest-performing list. 

JFTK looks at math and reading as individual measures of performance 
and allows schools to be recognized for their achievement independently from 
other curricular areas. Currently, math and reading are the focus of NCLB 
accountability, but we must remember that science assessments will be 
required beginning in 2007. At that time it will be important to also recognize 
schools doing well in science. By breaking performance criteria down to 
individual subject areas we get a more detailed look at what content areas 
schools are both succeeding in and struggling in. When compared over time we 
can begin to evaluate the effectiveness of new curricula or increases in 
resources aimed at improving a specific subject area. This is important 
information for every school and district responsible for implementing 
improvement strategies. 

A drawback of looking at each subject area independently is analogous to 
“missing the forest for the trees.” We want our students to be well rounded in 
their educational abilities because one who can do the math but fails to be able 
to express their answer or apply that mathematical process to real-world 
applications will not succeed. Additionally, using one year of data for such 
analysis means that the results are influenced by the yearly variation inherent 
in testing different groups of students. Another drawback of breaking data 
down to each subject area is the potential for misinterpretation of the resulting 
lists. For example, it is important to recognize that the number of schools that 
are strong performers in each subject area is not a measure of performance 
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between subject areas. While there are 23 schools considered strong- 
performers in math only and 7 schools considered strong-performers in reading 
only, reading scores across the state are higher than math scores. The higher 
number of schools considered strong-performers in math shows that the 
variance in math scores is less than the variance in reading scores. More 
schools are within the 5% opportunity gap in math than in reading. 

Using three year average composite scores takes a broader approach to 
defining higher-performing such that higher-performing high schools are those 
who consistently score higher than the state average both over time and as a 
composite of all 4 curricular areas tested (math, reading, writing and science). 
The CEPARE criteria also look for higher performance of all students within the 
school, requiring that groups both advantaged and disadvantaged students are 
doing 1/3 of a standard deviation better than the state average of their 
advantaged and disadvantaged counterparts. In addition, higher-performing 
schools must do 1/3 of a standard deviation better than predicted. This 
prediction is based on a regression analysis and the resulting standardized 
residual for the following variables: composite scale score for 1 1 th graders, 
composite scale score for 8 th graders, percent free or reduced lunch, and 
percent of the population with a bachelor’s degree. Meeting all of these criteria 
is difficult and the resulting list of higher-performing schools is much shorter 
than that of JFTK. In a sense, CEPARE raises the bar. What we miss in terms 
of individual subject area analysis we pick up in our analysis of the scores of 
advantaged and disadvantaged students. This disaggregation of scores fits with 
the mandates of NCLB requiring that “no child- regardless of his or her 
background — is left behind” (U.S. DOE, 2002). 

Combining these definitions, or looking at the overlap between the 
resulting list gives us a list of those schools may be considered the highest- 
performing. These schools are consistently scoring above the state average for 
all of their students, scoring higher than expected, and near the top of their 
comparable groups. 
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Knowing where the schools lie may help in determining appropriate 
higher-performing/ lower-performing school relationships. While initial analysis 
shows that while there may be a higher concentration of higher-performing 
schools in the southern Maine region, there are higher-performing (by some 
definition) schools throughout the state. It appears that higher-performing 
schools are generally located in the more populous portions of various regions 
throughout the state. 

List Discrepancies Explained 

To understand how one school may be considered a strong performer in 
both math and reading yet not meet the CEPARE high-performing criteria it is 
helpful to look at a specific example. Big school A is in a comparable group of 
19 schools, and is within 5% of weighted average of the three top schools in 
that pool in regard to both math and reading scores. Therefore, it is considered 
a strong-performer by JFTK. However, when we examine the three-year 
composite scale score for that school we see a score that is almost a third of a 
standard deviation below the state average scale score. Big School A had a 
score of 531.5 while the state average score was 532.4 with a standard 
deviation of 3.3. In a sense this school only looks like a higher-performing 
school because the others in its comparison group are doing so poorly. Figure 2 
shows how the top comparable schools in Big School A’s group compare to the 
top performers in the state. The composite scale score for Big School A led to a 
designation of neither higher nor lower performing from CEPARE. 
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Figure 2. 



■ Exceed; I Meets & Above 




All Tested Students Top Compaidble 

Schools 



Important Note: This shows the achievement of all tested students regardless of how long they have been at the 
school. The achievement of students who have been enrolled since the school's lowest high school grade is a better reflection of a 
school's academic program and will become available when the state provides the data. 

See 'JFTK Chart Features' for more detail. 




Important Note: This shows the achievement of all tested students regardless of how long they have been at the 
school. The achievement of students who have been enrolled since the school's lowest high school grade is a better reflection of a 
school's academic program and will become available when the state provides the data. 

See 'JFTK Chart Features' for more detail, 



Lower Performing Schools 

By using a similar process CEPARE has also defined a lower-performing 
list of high schools in Maine. Just For The Kids does not do a lower-performing 
analysis. CEPARE uses the same objective performance measures to determine 
what schools in the state are lower-performing. The lower-performing criteria 
require that groups of both advantaged and disadvantaged students are doing 
1 / 3 of a standard deviation below the state average of their advantaged and 
disadvantaged counterparts. In addition, lower-performing schools must do 
1/3 of a standard deviation lower than predicted. We believe that it is 
important to identify lower-performing schools in addition to the higher- 
performing schools. It may motivate those schools to improve and allows for 
resources to be allocated to those schools that need them most. Creating lists 
of lower performing schools must be done and reported in sensitive ways that 
encourage schools to improve and move into discussions about how to improve 
without being defensive. The list should be seen as identifying those schools 
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most in need of help. In order to use the lower-performing list in positive ways 
the measures of performance must be as objective as possible and resources 
must be provided to designated schools. Each lower-performing school should 
be supported as it builds on positive aspects within the school. This support 
may include working with a higher-performing mentor schools to improve 
achievement. 

Target Audience 

School level performance reports are part of the public domain and as 
such are accessed by a variety of different individuals and organizations. 
Educators, administrators, parents, students and policy makers all have a 
stake in the outcomes of studies on school performance. With this in mind it is 
crucial to be clear about the objectives of such studies and the intended 
audience of each. No one wants to hear that the school they attend, work for, 
pay for or represent is lower performing. Yet to allow schools to continue to fail 
their students is in no one’s interest. The goals of defining and applying best 
practices should be clear and the limitations of school performance studies 
should be clearly presented. 

Pairing Similar Schools 

If we can pair higher-performing and lower-performing schools in 
mentorship-model relationships and provide the resources necessary for 
improvement we can improve achievement outcome for those students in the 
state who need the most help. By pairing lower-performing schools with higher- 
performing schools of similar size, economic contexts and geographic regions 
we show that success is possible for a given school context and we can 
demonstrate how to achieve that success. 

What about the schools in the middle? The question we have to ask 
ourselves is whether or not average is good enough. I would suggest that it is 
not. At this point we have a long way to go toward all students meeting grade 
level benchmarks. Even the highest-performing schools in Maine still have high 
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percentages of students not meeting the benchmarks. As the lower-performing 
schools begin to improve there will be a natural shift upward in the state 
average scores; however we also have to expect more from even the higher- 
performing schools. There need to be achievement goals and increased 
expectations set forth for all schools (this relates to NCLB annual yearly 
progress-AYP). All schools need to focus on closing the achievement gaps 
between students. There will always be higher and lower performing schools in 
relation to state averages but hopefully someday, even the relatively lower- 
performing schools will be highly successful. 

Closing Thoughts 

Good assessments of school performance, like all good assessments, 
should follow a multi-source, multi-method assessment model. We have 
explored two higher-performance school definitions but other definitions may 
add to our understanding of what a truly high performing school looks like. 

The key is to understand what each definition brings to the discussion, what 
commonalities exist between definitions and what the differences are. As we 
continue our studies here at CEPARE we will be adding the financial aspects of 
higher performing schools and then focusing on topics such as school climate. 
We hope this research provides direction for both policy makers and educators 
as we work to improve educational outcomes for all Maine students. 
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